Deep Poetry: Word-Level and Character-Level Language Models for Shakespearean Sonnet Generation

Text generation is a foundational task in natural language processing, forming the core of a diverse set of practical applications ranging from image captioning and text summarization to question answering. However, most of this work has focused on generating prose. We investigate whether deep learning systems can be used to synthesize poetry, in particular Shakespearean-styled works. Previous work on generating Shakespeare prose involved training models exclusively on the word or character level. Here, we implement those previous models for poetry generation and show that models that combine word and character level information, such as a Gated LSTM and a CNN-based LSTM, significantly outperform the baseline word-LSTM and char-LSTM models. Perplexity scores for the two complex models are almost 10 fold better than that for our baselines, and human ratings of the model-generated sonnets reflect this as well. In particular, the sonnets our complex models generate have a coherent meaning and relatively correct meter without blatantly copying Shakespeare’s original works. These results encourage us that models that blend word and character level information would be useful for a variety of tasks outside of just poetry generation and may be crucial in bridging the gap between computer generated and human written text.